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(54) Abstract Title 

Controlling an electronic device by detecting a handheld member with a camera 



(57) A means for controlling an electronic device 1 comprises a control member 4 adapted to be held in a 
users hand and moved in space. The control member 4 has at least one characteristic such as the shape of 
spokes 8,9,19,11 or coloured marks such as circle, square, cross and triangle 12,13,14,15 enabling its position 
and orientation in space to be determined. A camera means 5 detects the position of the control member 4 
with respect to time and an analysing means 6 analyses the positional information to ascertain the movement 
of the control member 4. Control signal generating means 7 generates signals for the electronic device 1 in 
response to the analysed movement of the control member 4. The camera means may be a stereoscopic array 
of pinhole cameras, or a single camera; the member 4 may have a ring (23, fig 4) connecting the spokes; while 
the analysing means 6 may be hardware or software. Comparison with a reference image of the control 
member may be made. The means may join to a games console (1, fig 3) in place of a conventional joystick or 
keypad, or the camera means may be integral to the console. 



15 




> 



5/11/2009, EAST Version: 2.3.0.3 



1/1 




5/11/2009, EAST Version: 2.3.0.3 



2379493 

1 

CONTROLLING ELECTRONIC DEVICES 

This invention relates to means for controlling electronic devices such as 
games consoles or the like. 

5 

Conventionally such electronic devices are controlled by a joystick or 
keypad, usually integrated into the device. The device may have an 
integrated display screen, or be connected to a separate display monitor, 
which displays the game being played. Movement of the joystick or 
10 depression of buttons on the keypad sends control signals to the device. 
One disadvantage of the joystick and the keypad is that they may not be 
easy to use, especially for users with disabilities affecting the fine motor 
control of the hands. 

15 According to the present invention, a means for controlling an electronic 
device comprises a control member adapted to be held and moved in 
space by a user, and having at least one characteristic enabling its 
orientation in space to be determined, a camera means for detecting the 
position of the control member with respect to time, means for analysing 

20 the positional information to ascertain movement of the control member, 
and means for generating control signals for the electronic device in 
response to the movement of the control member. 

The position of the control member at any point in time is detected by the 
25 camera means. The orienting characteristic on the control member means 
that movement can be deduced from the positional information and 
translated into control signals for the device. The control member has a 
greater freedom of movement than the normal joystick or keypad. For 
example, it can be moved bodily in space up or down, side to side, and 
30 towards or away from the camera means. It can also be rotated. Each of 
these movements can be detected and used to generate control signals for 
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the device. Where the device is a games console, different movements 
can be used to move a character and perform other functions such as 
selecting a mode of firing a weapon. The control member is easy to use, 
as it does not rely significantly on fine motor control. 

5 

The control member may comprise at least two spokes radiating from a 
central axis, with at least one spoke having an orienting characteristic. 
Preferably four spokes are provided, although there could be as many as 
eight. 

10 

The orienting characteristic is preferably provided on the outer end of at 
least one spoke. Preferably, the orienting characteristic comprises at 
least one distinguishing shape or mark. Each orienting characteristic may 
comprise a mark such as a cross, circle, triangle or square. The orienting 
15 characteristic may be the shape of the spoke. For example, one spoke 
could be longer than the rest, or of a different outline. Different colours 
may also be used as well as other distinctions. 

The outer ends of the spokes may be joined by a ring. The control 
20 member may also include a handle on its rear side. 

The preferred control member with four spokes enables rotation of the 
control member about the central axis (i.e. in the plane of the spokes) to 
be detected, as well as inclination of the spokes to the plane. 

25 

Conveniently, the analysing means use algorithms to deduce the 
movement of the control member from the positional information acquired 
by the camera means. The analysing means may be executed as software, 
or hardware such as microprocessors. 

30 
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The signal generating means uses the movements analysed by the 
analysing means to generate appropriate control signals for operating the 
device. The generating means is preferably executed as software. It may 
generate signals which emulate those of existing game consoles, enabling 
5 the control member to be used with existing game consoles. 

The camera means preferably comprises a stereoscopic camera array, 
having a pair of pinhole cameras whose optical properties are known. 
The images from the two cameras are combined to provide an accurate 
10 representation of the position of the control member in space, so that its 
movement can also be accurately analysed. 

Alternatively, one camera may be used. Again, its optical properties are 
known. With one camera, movement towards and away from the camera 

15 is more difficult to analyse than with two. However, large movements 
towards or away from the camera can be detected, by looking at the 
change in surface area, and comparing the current dimensions of the 
control member with those for a given distance from the camera, which 
will be known. The other movements (up or down, side to side, rotation 

20 and inclination) can be determined accurately by comparing the camera 
image with an ideal image of the control member at a fixed position. 

Preferably, the analysing means stores the size and shape of the control 
member. This enables more accurate analysis of the movement of the 
25 control member to be made. The camera image can be compared with the 
stored ideal image to determine the location and orientation of the control 
member. 

The camera means and the analysing and generating means are preferably 
30 combined into a single unit. This may be integrated into a games 
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console. The camera means and the analysing and generating means may 
be separate and adapted to be joined to a console by appropriate leads. 

Various embodiments of the invention are illustrated, by way of example 
5 only, in the accompanying drawings in which: 

Figure 1 is a schematic illustration of a control means for an 
electronic device; 

10 Figure 2 shows the control member in detail; 

Figure 3 is similar to Figure 1, but shows a modification; and 

Figure 4 shows a modified control member. 

15 

Figure 1 shows a schematic view of a means for controlling an electronic 
device such as a games console 1. The games console 1 is connected by 
appropriate leads 2 to a monitor 3 on which the game being played is 
displayed. 

20 

The control means has a control member 4 which is held and moved in 
space by a user (not shown) and has characteristics enabling its 
orientation in space to be determined. A camera means 5 detects the 
position of the control member 4 over time, analysing means 6 analyses 
25 the positional information to ascertain the movement of the control 
member 4, and generating means 7 generates control signals for the 
device 1 in response to movement of the control member 4. 

The control member 4 is shown in more detail in Figure 2. It comprises 
30 a plastics member having four spokes 8, 9, 10, 11 radiating in a cross- 
shape from a central axis A. Each spoke has on its respective outer end 
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of its front face an orienting characteristic 12, 14, 14, 15. The orienting 
characteristics are a different colour from the spokes, and each has a 
different shape. In this case they are a cross, a square, a triangle, and a 
circle, but it will be appreciated that any suitable shapes could be used. 
5 The control member 4 also has on its rear face a handle (not shown) by 
which it can be held by the user. 

The control member 4 can be moved bodily in space, and relative to the 
camera means 5, in the following ways: up or down, side to side, and 
10 towards or away from the camera means 5. In addition, it can be rotated 
about its central axis A (in the plane of the spokes 8, 9, 10, 11) or 
inclined to that plane. 

The camera means 5 detects the position of the control member 4 as it is 
15 moved. The camera means 5 is mounted on the console 1, and comprises 
a stereoscopic camera array of two pinhole cameras 13,14 whose optical 
properties are known. The images from the cameras 13, 14 are analysed 
by the analysing means 6, to ascertain the movements of the control 
member 4. 

20 

The analysing means 6 is integrated into the console 1 and is executed as 
hardware, such as a microprocessor, although it may be provided as 
software. The analysing means 6 receives the images from the cameras 
13, 14, and processes these to determine the position and orientation of 

25 the control member 4 at any given time. The use of the stereoscope 
camera array means that the location of the control member 4 in space 
can be determined absolutely. The analysing means 6 uses suitable 
algorithms to process the images, and holds an ideal image of the control 
member 4 at a given distance from the camera means 5 and with a given 

30 orientation, to assist in the processing. Comparison of one set of images 
with the previous one is used to determine movement. Thus, changes in 
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the position of the orienting characteristics in the plane of the spokes are 
used to determine the up or down and side to side movement of the 
control member 4, while changes in their size are used to determine the 
towards and away movement. Angular movement of at least two of the 
5 orienting characteristics is used to determine rotation, and changes in 
their shape determine inclination. 

The movement ascertained by the analysing means 6 is used by the 
generated means 7 to provide control signals for the console 1. The 

10 generating means 7 is integrated into the console 1, and is executed as 
software. The generating means 7 assigns each movement (as described 
above) a particular command to the games console 1 - for example 
movement of a character in the game, selection of a mode or the firing of 
a weapon. The ability to rotate the control member 4 is particularly 

15 useful when the game simulates the driving of a vehicle. 

Thus, simple movements of the control member 4 can be used to send 
control signals to the games console 1, ensuring ease of use. 

20 Figure 3 shows a modification, where the games console 1 is separate 
from the control unit 20. In Figure 3 the camera means 5, analysing 
means 6 and generating means 7 are all housed in the control unit 20, 
which is connectable to the games console 1 by appropriate controller 
leads 21. The generating means 7 are arranged to generate control 

25 signals which emulate those of an existing game controller (not shown), 
such as a joystick or keypad. 

Further, in Figure 3 the camera means 5 comprises a single camera 22. 
This, naturally, does not provide as much information as the pair of 
30 cameras 13, 14 of Figure 1, as the position in the towards and away from 
camera direction is more difficult to detect. However, the camera image 
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can be compared with the ideal image, and the difference in the surface 
area and size will give a good approximation of towards / away from 
distance. The other movements can be determined accurately by 
comparison with the ideal image, and comparison of one camera image 
5 with a previous one. Otherwise, the embodiment of Figure 3 is the same 
as that of Figure 1 . 

Figure 4 shows another embodiment of the control member 4, in which a 
ring 23 connects the outer ends of the spokes 8, 9, 10, 11. Otherwise the 
10 member 4 of Figure 4 is the same as that of Figure 2, and corresponding 
reference numerals have been applied to corresponding parts. The ring 
23 has advantages as it provides a further comparison item for processing 
of the image. 

15 It will be appreciated that the control member 4 could be modified in 
other ways (not shown). For example, only one spoke need have the 
orienting characteristic, but more orienting characteristics increases 
accuracy. Alternatively, the spokes could be of different shapes, so that 
the shape provides the orienting characteristic. While the control 

20 members 4 shown have four spokes, any number from two to eight would 
be possible. Whatever the size and shape of the control member 4, its 
dimensions must be held in the analysing means 6 to enable the analysis 
of its movements to be made. 

25 



30 
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CLAIMS 



1. A means for controlling an electronic device comprises a control 
member adapted to be held and moved in space by a user, and having at 

5 least one characteristic enabling its orientation in space to be determined, 
a camera means for detecting the position of the control member with 
respect to time, means for analysing the positional information to 
ascertain movement of the control member, and means for generating 
control signals for the electronic device in response to the movement of 
10 the control member. 

2. A means for controlling an electronic device as claimed in claim 1, in 
which the control member comprises at least two spokes radiating from a 
central axis with at least one spoke having an orienting characteristic. 

15 

3. A means for controlling an electronic device as claimed in claim 2, in 
which the control member comprises four spokes radiating from a central 
axis with at least one spoke having an orienting characteristic. 

20 4. A means for controlling an electronic device as claimed in claim 2 or 
claim 3, in which the orienting characteristic of the control member is 
located at an outer end of at least one spoke. 

5. A means for controlling an electronic device as claimed in any 
25 preceding claim, in which the orienting characteristic of the control 

member is at least one distinguishing shape or mark. 

6. A means for controlling an electronic device as claimed in claim 2 or 
claim 3, in which the orienting characteristic of the control member is the 

30 shape or size of at least one spoke. 
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7. A means for controlling an electronic device as claimed in any of 
claims 1 to 3, in which the orienting characteristic is the colouring of the 
control member. 

5 8. A means for controlling an electronic device as claimed in any of 
claims 2 to 7, in which the spokes of the control member are joined by a 
ring. 

9. A means for controlling an electronic device as claimed in any 
10 preceding claim, in which a handle is provided on the rear side of the 

control member. 

10. A means for controlling an electronic device as claimed in any 
preceding claim, in which the analysing means uses algorithms to deduce 

15 the movement of the control member from the positional information 
acquired by the camera means. 

11. A means for controlling an electronic device as claimed in any 
preceding claim, in which the analysing means is executed as software. 

20 

12. A means for controlling an electronic device as claimed in any 
preceding claim, in which the analysing means is executed as hardware. 

13. A means for controlling an electronic device as claimed in any 
25 preceding claim, in which the signal generating means uses the 

movements analysed by the analysing means to generate appropriate 
control signals for operating the electronic device. 

14. A means for controlling an electronic device as claimed in claim 13, 
30 in which the control signal generating means is executed as software. 
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15. A means for controlling an electronic device as claimed in any 
preceding claim, in which the camera means comprises a stereoscopic 
camera array, having a pair of pinhole cameras whose optical properties 
are known. 

16. A means for controlling an electronic device as claimed in any 
preceding claim, in which the camera means comprises a single camera 
whose optical properties are known. 

17. A means for controlling an electronic device as claimed in any 
preceding claim, in which the analysing means stores the size and shape 
of the control member. 

18. A means for controlling an electronic device as claimed in any 
preceding claim, in which the camera means and the analysing and 
generating means are combined into a single unit and integrated into a 
games console. 

19. A means for controlling an electronic device as claimed in any 
preceding claim, in which the camera means and the analysing and 
generating means are separate and adapted to be joined to a games 
console by appropriate leads. 

20. A means for controlling an electronic device substantially as 
described herein with reference to and as illustrated in Figures 1 to 4 of 
the drawings. 
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